1. Field of the Invention
The present invention relates to an electronic device comprising a digital reverberator, to a computer-readable memory storing a digital reverberator, and to a method of digital sound reproduction using a digital reverberator for a user of an electronic device.
2. Description of Related Art
Advances in computational hardware have made virtual reality (VR) a ubiquitous technology, especially in computer games (many of which are now being released in ‘3D’ to further enhance realism). The user-perceived realism of a virtual reality application depends on its design, which must take into account several different output modalities such as visual, auditory, and tactile. Along with realistic graphics rendering, spatial audio is one of the most important factors that affect how realistic its users perceive a virtual environment or a computer game [see M. Zyda, “From visual simulation to virtual reality games”, Computer, vol. 38, no. 9, pp. 25-32, September 2005, the contents of which are fully incorporated herein by reference]. While spatial audio is well-established, especially in music reproduction and movies, VR applications and computer games have been relatively slow in taking up the available technology. This is especially so with room acoustics simulators, also known as digital reverberators: such reverberators are computer-implemented and are able to determine how a sound source is heard in an enclosure, such as a room, a maze, or a hall. The reason for this slow uptake is the computational complexity of accurate room acoustics simulators which make interactivity infeasible, if not impossible on consumer-grade electronic devices, such as gaming machines including, but are not limited to, the XBOX 360®, PLAYSTATION 3®, desktop PCs, laptops, notebooks, tablets (e.g. IPAD®), and Smartphones. Therefore, the spatial audio effects that can be provided by conventional electronic devices have remained rather rudimentary.
The primary aim of a digital reverberator is to emulate the response of a virtual room given the parameters such as room geometry, source and receiver positions and directivity patterns, and absorption and diffusion characteristics of room surfaces [see M. Vorlander, Auralization—Fundamentals of Acoustics, Modeling, Simulation, Algorithms and Acoustic Virtual Reality. Springer-Verlag, 2008, the contents of which is fully incorporated herein by reference]. This process is known as ‘room auralization’. A room is a multipath environment and a room impulse response (RIR) typically consists of the direct path, early reflections, and late reverberation. For a listener positioned in a room, the early portion of RIR determines the perceived width of the source and the late portion determines how distance of a sound source and the size of a room are perceived. If multiple receivers are used, RIRs can also inform a listener about the relative direction of a sound source.
Artificial reverberation is a technique that has been used to improve the spatial sound quality of a recorded sound for a user. Initially the reverberation was applied to the sound as it was recorded. For example, the earliest artificial reverberators were electromechanical devices such as plates or springs and were used to improve spatial realism of studio recordings prior to radio broadcast. However, advances in digital systems and digital signal processing algorithms opened up the possibility of applying digital artificial reverberation (or more simply ‘digital reverberation’) to a sound as it was played back to the user. In this way it was possible to change the user's perception of the sound, and this became particularly important for imparting realism to computer games and virtual reality, especially games of the first-person shooter type.
One of the most well-known digital artificial reverberators was Schroeder reverberator [see for example M. R. Schroeder and B. F. Logan, “Colorless artificial reverberation,” IRE Transactions on Audio, vol. AU-9, pp. 209-214, 1961] which consisted of parallel comb filters and allpass filters connected in cascade. Comb filters model frequency modes of a room transfer function and allpass filters increase the reflection density which is necessary to obtain a rich reverberation tail. Whilst Schroeder reverberators generate colorless reverberation, they also cause metallic ringing artifacts, especially for long reverberation times. This issue was addressed by Moorer [see J. A. Moorer, “About this reverberation business,” Computer Music Journal, vol. 3, no. 2, pp. 13-28, 1979] by inserting a lowpass filter in the feedback path of comb filters. However, neither Schroeder nor Moorer reverberators allow explicit control over the frequency-dependent reverberation time. This problem was addressed by Gardner [see W. G. Gardner, “Reverberation algorithms,” in Applications of Digital Signal Processing to Audio and Acoustics, M. Kahrsand K. Brandenburg, Eds. Boston, Mass., USA: Kluwer Academic, 1998, pp. 85-131] by adding a global lowpass feedback path to the cascade of allpass filters.
One of the most commonly used digital artificial reverberators today is the feedback-delay network (FDN) [see J. Stautner and M. Puckette, “Designing multichannel reverberators,” Computer Music J., vol. 6, no. 1, pp. 52-65, 1982, the contents of which is fully incorporated herein by reference] that was proposed as a multichannel extension of the Schroeder reverberator. FDN reverberators consist of parallel delay lines connected recursively over a unitary (i.e. energy preserving) feedback matrix. FDN-type reverberators were improved by Jot and Chaigne [see J.-M. Jot and A. Chaigne, “Digital delay networks for designing artificial reverberators,” in Proc. 104th Conv. Audio Eng. Soc., Preprint #3030, Paris, France, 1991, the contents of which is fully incorporated herein by reference], who used delay lines connected in series with absorptive filters, and a global tonal correction filter to obtain a frequency-dependent reverberation time. Investigation of matrices that provide maximum scattering is a matter of on-going research [see C. Faller and F. Menzer, “Unitary matrix design for diffuse jot reverberators,” Presented at 128th Audio Engineering Society Convention, Preprint #7984, May 2010]. Another type of artificial reverberator related to FDN is based on the concept of digital waveguides, and is called the digital waveguide network (DWN) [see J. O. Smith, III, “A new approach to digital reverberation using closed waveguide networks,” in Proc. 11th Int. Comput. Music Conf., Burnaby, BC, Canada, 1985, pp. 47-53, the contents of which is fully incorporated herein by reference]. Such reverberators were more formally investigated by Karjalained et. al. [see M. Karjalainenet al. “Digital Waveguide Networks for Room Response Modeling and Synthesis,” in Proc. 118th Conv. Audio Eng. Soc., Preprint #6394, Barcelona, Spain, 2005, the contents of which is fully incorporated herein by reference] for rectangular enclosures.
Recent advances in multicore processors, and more particularly general purpose graphical processing units (GPGPU) have made it possible to use full scale, real-time, interactive room simulators in computer games and VR applications. However, such algorithms are still impractical for all kinds of consumer-grade electronic devices, and especially low-cost and low-power portable terminals such as, but not limited to: tablet computers, portable game consoles and mobile/smart phones.
For example, a full scale VR suite such as a CAVE Automatic Virtual Environment (CAVE) can use Ambisonics [see M. A. Gerzon, “Ambisonics in multichannel broadcasting and video,” J. Audio Eng. Soc., vol. 33, no. 11, pp. 859-871, November 1985, the contents of which is fully incorporated herein by reference] or wave field synthesis (WFS) which requires from tens to hundreds of loudspeaker channels [see M. M. Boone, U. Horbach, and W. P. J. Bruijn, “Spatialsound-field reproduction by wave-field synthesis,” J. Audio Eng. Soc., vol. 43, no. 12, pp. 1003-1012, December 1995]. In contrast, a portable game console will have two channel audio output which may be used to output binaural audio [see H. Møller, “Fundamentals of binaural technology,” Appl. Acoust., vol. 36, no. 3-4, pp. 171-218, 1991, the contents of which is fully incorporated herein by reference] over headphones. On the other hand a typical home user will probably use a 5.1 or a 7.1 audio system or more possibly a stereophonic system with two channels of audio output.
As mentioned above, an artificial digital reverberator was proposed by Smith [see J. O. Smith, III, “A new approach to digital reverberation using closed waveguide networks,” in Proc. 11th Int. Comput. Music Conf., Burnaby, BC, Canada, 1985, pp. 47-53].
U.S. Pat. No. 4,984,276 (also to Smith) discloses is a signal processor formed using digital waveguide networks. The digital waveguide networks have signal scattering junctions. A junction connects two waveguide sections together or terminates a waveguide. The junctions are constructed from conventional digital components such as multipliers, adders, and delay elements. The signal processor is typically used for digital reverberation and for synthesis of reed, string or other instruments.
These digital reverberators were further developed by Karjalainen et al. [see M. Karjalainen, P. Huang, and J. O. Smith, “Digital Waveguide Networks for Room Response Modeling and Synthesis,” in Proc. 118th Cony. Audio Eng. Soc., Preprint #6394, Barcelona, Spain, 2005] in order to produce a maximally efficient simulation of acoustic space. Karjalainen et al. proposed a so-called ‘sparse’ digital waveguide network having one (or a few) scattering junction per simulated wall; the scattering junctions are interconnected by digital waveguides (i.e. a respective bi-directional delay line). The network is lossless, except for wave digital admittances applied at each scattering junction. The digital admittances start and terminate on a respective scattering junction and therefore appear diagrammatically as a loop. Whilst the sparse network is reasonably computationally efficient (reported as taking 15% of CPU time on a 1 GHz G4 PowerPC process with single simulated source and receiver), the authors admit that tuning of model parameters involves much guesswork, and relies on actually listening to the simulated room, adjusting parameters and listening again; this process is repeated until a satisfactory sound simulation is perceived.
This is a significant problem because it prevents the sparse digital waveguide network being used in all but the simplest of cases. For example, many interactive computer games involve tens or hundreds of acoustic spaces (rooms, street scenes, etc.). It is completely impractical for computer game developers to manually configure each space for all possible positions of source and receiver (e.g. player position within the space).
It is apparent that there is a need for a digital reverberator, for example of the sparse digital waveguide network type, but which can be readily implemented in computer game and virtual reality environments, without the need to manually tune model parameters for each room. Furthermore, it would be preferable if such a reverberator is operable in real-time on consumer-grade computer devices so that other critical game or simulation features (e.g. graphics processing) are not unduly compromised.